Generalisation in Feedforward Networks
نویسندگان
چکیده
We discuss a model of consistent learning with an additional restriction on the probability distribution of training samples, the target concept and hypothesis class. We show that the model provides a significant improvement on the upper bounds of sample complexity, i.e. the minimal number of random training samples allowing a selection of the hypothesis with a predefined accuracy and confidence. Further, we show that the model has the potential for providing a finite sample complexity even in the case of infinite VC-dimension as well as for a sample complexity below VC-dimension. This is achieved by linking sample complexity to an "average" number of implement able dichotomies of a training sample rather than the maximal size of a shattered sample, i.e. VC-dimension.
منابع مشابه
Generating-shrinking algorithm for learning arbitrary classification
This paper proposes a novel generating-shrinking algorithm, which builds and then shrinks a three-layer feed-forward neural network to achieve arbitrary classification in n-dimensional Euclidean space. The algorithm offers guaranteed convergence to a 100% correct classification rate on training patterns. Decision regions resulting from the algorithm are analytically described, so the generalisa...
متن کاملDependency Parsing with LSTMs: An Empirical Evaluation
We propose a transition-based dependency parser using Recurrent Neural Networks with Long Short-Term Memory (LSTM) units. This extends the feedforward neural network parser of Chen and Manning (2014) and enables modelling of entire sequences of shift/reduce transition decisions. On the Google Web Treebank, our LSTM parser is competitive with the best feedforward parser on overall accuracy and n...
متن کاملNeural Networks for Breast Cancer Diagnosis
Breast cancer diagnosis has been approached by various machine learning techniques for many years. This paper describes two neural network based approaches to breast cancer diagnosis, both of which have displayed good generalisation. The first approach is based on evolutionary artificial neural networks. In this approach, a feedforward neural network is evolved using an evolutionary programming...
متن کاملExtreme Learning Machine: A Review
Feedforward neural networks (FFNN) have been utilised for various research in machine learning and they have gained a significantly wide acceptance. However, it was recently noted that the feedforward neural network has been functioning slower than needed. As a result, it has created critical bottlenecks among its applications. Extreme Learning Machines (ELM) were suggested as alternative learn...
متن کاملEvolutionary Multi-task Learning for Modular Training of Feedforward Neural Networks
Multi-task learning enables learning algorithms to harness shared knowledge from several tasks in order to provide better performance. In the past, neuro-evolution has shown promising performance for a number of real-world applications. Recently, evolutionary multitasking has been proposed for optimisation problems. In this paper, we present a multi-task learning for neural networks that evolve...
متن کاملIntroduction to multi-layer feed-forward neural networks
Basic definitions concerning the multi-layer feed-forward neural networks are given. The back-propagation training algorithm is explained. Partial derivatives of the objective function with respect to the weight and threshold coefficients are derived. These derivatives are valuable for an adaptation process of the considered neural network. Training and generalisation of multi-layer feed-forwar...
متن کامل